Mozilla Speaker Series [January ‘17] 
Data and People: A Discussion with Laszlo Bock, Sr. Advisor and former SVP of People 
Operations, Google 


Hi everybody. Welcome. Thank you for coming to our monthly speaker series today. 
I'm Larissa Shapiro. | lead diversity and inclusion here at Mozilla and I'm super, 

Larissa Shapiro: super excited because we have Laszlo Bock here with us today and you probably 
know all about him, and I'm going to tell you a tiny bit anyway. then, let him say his 
own words. Thank you so much for coming. Laszlo led Google's people operations 
from 2006 to 2016. In that role, he was responsible for attracting, developing, 
retaining and delighting Googlers. He believes that giving people freedom and 
supplementing our human instincts with hard science are steps on the path to 
making work meaningful and people happy which is something that we also believe 
in part of why we asked him to come. 


During Laszlo's tenure at Google, Google was named the Best Company to Work 
more than 30 times and received over 100 awards as an employer of choice. | think 
many of us will remember. He's also the author of WORK RULES! Insights from 
Inside Google to Transform How You Live and Lead, which was named one of the 
top 15 business books of 2015. | really love that he donates all the proceeds from 
this book to organizations that support children and veterans. Do | have that right? 


Laszlo Bock: Yes. 


Larissa Shapiro: Yeah. Local organizations like Peninsula Bridge which works with disadvantaged 
youth to provide better opportunities. Super awesome. He's also testified before 
congress and written for Wall Street Journal, PBS Newshour. He also was an actor in 
his earlier life. He was on Baywatch. | read all about him, and he used to hold the 
world record for largest simultaneous Greek Syrtaki dance along with 1,620 other 
people. You can tell us about that if you want. All right. That's enough introduction. 


Laszlo Bock: Yeah. That's way too much. Can you hear me on the mic? Thank you. 

Larissa Shapiro: Yeah. You sound great. 

Laszlo Bock: Yeah. 

Larissa Shapiro: Do you want to say anything before we dive in? 

Laszlo Bock: No. Thanks for keeping it short. That was good. 

Larissa Shapiro: Okay. Good. I'll keep it short. | want to dive right into the heart of our shared 


passion which I think is people and data and motivating cultural change within 
companies and cultural values. You started the group at Google called People 
Analytics. Can you tell us more about it? What led to its formation, etc.? 


Laszlo Bock: 


Diane: 
Larissa Shapiro: 


Laszlo Bock: 


Larissa Shapiro: 
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Sure. Actually, it reminds me. There was one thing | want to say. It's amazing and an 
honor to be here at Mozilla. | can't remember when | started using your browser 
but it was in response to all the lousy browsers that were out there, and it was just 
better and more secure and faster and it's just so important to get people choice 
and it's great to have all of you who worked in this organization working hard to 
keep the internet accessible and give people a product that's fantastic, that's 
available, that's focused on their privacy and just does all the right thing. It is for me 
really, really cool just to be here and to be able to breathe the same air that you're 
all breathing. | just want to say thank you for everything you've done to make the 
internet great and accessible, and it's really, really important. I'm really grateful. 


Thank you. 
Thank you. 


Thank you. People Analytics, the reason | decided to go into human resources, 
there were a couple of pieces of it, but one of them was that when I was in grad 
school, I've taken a couple of classes on statistics and operations. I'd always had an 
interest in biology and science and things like that. At the same time, I've always 
been frustrated by the way leaders behave and what they're saying, the disconnect 
between the values they espouse and what they actually live. | realized as | bumped 
along in my career that everyone in the people space, we make stuff up all the time 
like we think we're great managers, we think we're great at interviewing, we think 
we're great at building teams and we're better at it when we're like 50 or 60 years 
old than we are when we're 20 years old. 


All along the way, people are suffering because they have to work with us while we 
figure this stuff out and there's not a lot of science, not a lot of rigor. Before | joined 
Google, | read an article talking about how IBM had this 40-person analytics group 
dealing with some kind of issue or problem. | remember thinking, "Man, if you had 
have like 40 people who have advanced degrees, who are smart, who are 
thoughtful, if you can just focus on proving what works in the people space, that 
will be amazing because it's not been done." When | got to Google, one of the 
things I really wanted to do was try to build that out. We called it People Analytics. 
It was the first place to use that phrase and now, it's a whole thing. We started just 
by taking three analysts. There was a staffing analyst, an operations analyst and a 
compensation analyst. | said, "Okay. You three are now the People Analytics team," 
and they've grew from there. 


Awesome. Is it on? She's fixing mic. Okay. How did the People Analytics team set 
goals and evolve overtime? 


There were two elements. Part of it was there's kind of stuff that comes up, right? 
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How do we make staffing work more efficiently? When | joined Google, | think | had 
about 25 interviews, 20 or 25 interviews. The typical interview time was six to nine 
months before somebody got an offer, if they got an offer, most people didn't. | 
remember coming out here doing a housing search trying to figure out where | was 
going to rent because it's ... | don't know if you notice, it's crazy expensive out here. 
Everybody had a story like the realtor's husband had interview to Google, terrible 
experience, the taxi driver | was with, his cousin had interview to Google, terrible 
experience. 


It took forever, it was a bad experience. One of the real business problems that we 
started with was how do you make that better? How do you make it more efficient? 
How many interviews do you need to have? What do you need to ask and how do 
you need to run it? Not just to be more predictive but also give people a better 
experience. That was part of it. Figuring out like what's the biggest pain point and 
trying to fix it. The second part was just having interesting conversations with 
people on the team. | funded People Analytics by not putting headcount into other 
departments. 


For example, we didn't build a leadership development function for a long time and 
| used those heads instead to build People Analytics, and it was so that | could just 
sit down with the folks and we could just have interesting conversations. They knew 
way more than me about lots and lots of things and the Googlers, the employees 
would come with great things but it was always important to say that capacity so 
that we could just kind of blue sky and make stuff up and explore different 
directions. 


Can you tell us about an example of an experiment that team did, and/or some of 
the obstacles faced in trying to approach experimental HR that way? 


Yeah. The biggest one was something called Project Oxygen, and have lots of 
people heard about that? If they have ... Okay. Just a few. Great. If it was lots of 
people, I'll give you another example. Okay. That's what's important. Project 
Oxygen was basically the engineering culture with such that the view was managers 
are worthless. Management gets in your way, they slow you down, they never 
understand the job you're doing so you should have a few managers as possible. 
The team said, "Okay. We should tackle this question. We should definitively prove 
that managers are worthless and that they destroy value in organizations, or not 
destroy value." 


The thinking was if we can improve that, we can actually get rid of way more 
managers and we can run the company differently. What they found was actually 
managers are really important, and they segmented the company into teams that 
were high performing, and low performing. They correlated out with manager 
performance. You have this nice beautiful distribution of teams and people who 
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performed really well, who worked for people that we identified as good managers 
and people who were poorly performing and poorly performing teams working for 
people we identified as bad managers, but from a scientific perspective, you can't 
actually stop there as you know because you actually need a controlled experiment 
with random organization of all the variables. 


We thought it would be cruel to actually take people from the good managers and 
in a blinded way, assign them to terrible managers and see if they got so unhappy, 
they quit. We were lucky. This is one of the benefits of scale. One of the negatives, 
as you get bigger, it that it feels like a big company, it's more bureaucratic, but you 
actually can run a lot more experiments. What we saw was people were just 
randomly moving across, not randomly but people moved to all around teams on 
their own and they had no information about which managers were good or bad. 
They could ask people but they tended not to, and that wasn't reliable. We had a 
very natural experiment where we had this natural movement of people all over 
the company which appeared as best we can measure to be pretty random. 


What we found was when people left good managers, their performance declined, 
not just their happiness but they actually performed worst and when people who 
worked bad performance went to good managers, their performance and 
happiness actually improved. Then we figured out, "Okay. What are these managers 
doing?" We did a whole bunch of interviews, coded the interviews, did a bunch of 
factor analysis and the big insight was number one, managers do actually make a 
meaningful difference, but number two, management is not hard. Management is 
not like this mysterious science, it's actually kind of boring. 


There's eight things that the most effective managers that Google did, and we 
published that internally as a list. There's three things they did that were negative 
that they shouldn't do, and the eight things were things like have regular 
one-on-one meetings acted as a coach. What we then found was that if you ... You 
have this debate about do you want to have like change people's behaviors and 
beliefs or do you want to just instruct them what to do? You don't actually have to 
change people's belief systems, you can actually ... This may come up when we talk 
about diversity. 


We just give them much checklist. We said, "Here's the behaviors, we're going to 
survey your teams every six months." It's an anonymous survey. If you get more 
than three responses, we're going to show you the results and we're going to show 
you where your results are on these eight factors relative to the whole company 
and we get an aggregate favorability score of zero to 100%. What was interesting 
was we did that average favorability of managers with something like 57%. We run 
this thing every six months for two and a half years. At the end of that period, 
average favorability was on the order of 85%. 
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Laszlo Bock: 


What was crazy was as the people guy, | didn't have to build training. | didn't have 
to advance resources. It turns out if you let people know how they're doing in an 
objective non-threatening way that's not tied to consequence like pay or 
promotion, or what have you, most people actually want to improve and they will 
take steps to improve and they will get better. It's a really, really cool study rooted 
in, "Let's try to figure out something. We're totally wrong about that hypothesis. 
Okay. What do we do with it?" Then, how do you change behavior on a really large 
scale. It was fun. It was really good stuff. 


That's totally awesome. All right. | think I'm going to move to data and diversity 
because | think what you just said is so highly relevant, and | have so many 
questions about it because it's my passion but actually the Oxygen study is very, 
very relevant for us. Our organization has been focusing more intensively on 
inclusion and diversity for the last 18 months or so, and we did a big analytics 
phase, qualitative and quantitative. Then, we've built out a strategy and we're 
continuing to build. As | understand it, under your leadership, Google developed 
quite a number of resources and programs and did a lot of research around 
organizational impact of unconscious bias in particular. Can you tell us how that 
started and what you learned? 


Yeah. | read an article in New York Times, New York Magazine, something like that 
or New Yorker Magazine about an academic study and the study was roughly as 
follows. The experimental condition is you have people come in, and the person 
running the study hasn't read a bunch of stuff and tell them, "There's a task we 
need you to do. You're going to go into this next room. There's other people doing 
the task, and go do this thing." The people who were the experimental subjects 
were prompted in one of two ways. One of the ways they were prompted was to 
just have a control, "Here's a task. Go do this task." The other embedded within 
that was some explanations and some contextual clues that talked about how when 
human beings are confronted with people who are different than them, and that 
difference can take lots of forms, maybe a little uncomfortable, feels a little weird. 


What's interesting was they send in the two sets of people. In the room, there's a 
workspace and there is a black man sitting at a computer terminal doing some 
work. There's a chair next to the black man and the chair is a little too close for 
comfort. It's pretty close to where he's sitting. One of two things would happen. 
One set of experimental subjects would go in and they'd step forward, and they 
would see the chair, they'd pick it up, then move in a few inches over, sit down and 
get to work. The other set of people would come in, pause, look at the dude, sit 
down in the chair and start working. 


The finding in the study was that if you cue people to be aware of differences, their 
behavior changes in very small, subtle ways. They are more accepting of those 
differences and not to read too much into the study, but | took that back to actually 
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the People Analytics team and the diversity team and | said, "If this is legit, if this is 
quality science, what it suggests is ..." One of the ways to address this question of 
inclusion which then has all these mediating things which actually drive higher 
performance, but there's a couple of steps you can get there, "is not to win the 
hearts and minds of everyone." It's really hard to change belief systems, right? 
People are raised with all the stuff, our brains can only process a tiny bit of the 
information we actually take it at any one time. There's all these issues of cognitive 
load and salients. 


If you structure the environment a little bit differently and give a little nudge, 
maybe you can have a really disproportionate impact on how people interact just 
by raising awareness. We started, | don't know, four or five years ago. | think we're 
probably the first tech companies certainly to look at this question of unconscious 
bias and inclusion. We weren't the first to be public about it because | wanted to 
make sure it was a serious enough thing. | wanted to make sure if we put 
something out there, the science has got to be proven because like you guys, if you 
say Mozilla were doing this, people pay attention. 


If Google says we're doing whatever, people pay attention, so we had to get it right 
but the genesis of it was this notion that is there some small intervention you can 
make that will cause people to pause and be more aware that cause a difference in 
behavior rather than the more traditional approach which is, "We got to get 
everyone to believe. We got to change people's values. We got everyone to believe 
that everyone is equal and equally capable and should have the same opportunity." 
| think we should do that too, but that's a lot harder and my compromise was, "I'll 
just settle for having everyone behave as if they believe all those things." 


What kinds of small changes did you try? I'm super curious. 


We eventually built a course, and there's a public version available if you open up 
your Firefox browser, and use your search engine of choice and look for Google and 
unconscious bias. The search terms are Google and unconscious bias. There's a 
couple of videos. They're really rough but it actually has an early version of our 
training and it's an hour long program and a guy named Brian Welle was running it. 
There's also a website that we had put out called RE:WORK which is where we're 
trying to open source our HR ideas and programs. | guess they try to open source all 
the people stuff we're doing, and there's resources there. One of the things was 
quite simply an hour long training course and most training courses aren't effective. 
The outcomes and measurements of what happens overtime aren't done well, but 
this started from the way we got there and develop something that actually was 
proven to be effective was we started not from, "Let me tell you about diversity or 
let me tell you why inclusion is important," because when you start from that point, 
you trigger something called identity threat. 
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Nobody thinks of themselves as racist. Nobody thinks of themselves as sexist or 
homophobic or what have you, but everybody has bias. Every human being has 
bias. It's just how we relate to the world, right? We want to hang out with people 
who wear the same sports jerseys we were, if you are in the sports. The way you 
circumvent the identity of threat is you start with less threatening things like there's 
an online thing called the Implicit Association Test which is this word association. It 
shows that, "Yeah. There's a little bias in how I think about words and some words, 
| associate more with men and more with women." Even less threatening, it just 
start with optical illusions because we all believe we're rationale actors and we 
understand the world and we're navigating it thoughtfully and well and our 
cognitive dissonance forces us to do that in retrospect. We go through life, and 
looking back, it all makes sense as this is why we're going to have these decisions. 


If you look at optical illusions like there's a class going where there's three prongs 
or two prongs on the thing, that's your brain telling you, you actually can interpret 
reality. It's actually neither of those things. It's a line drawing and it's neither three, 
nor two but your brain will flip between one or the other. From that kind of entry 
point, you can get even hostile audiences. People who are hostile in the idea to go 
in, "Okay. You're right. | don't get that." From then, you can step up to things that 
eventually have a conversation about, "Okay. If somebody is transgendered, why 
are they just a human being just like me and why should they be treated just the 
same ways as I'd be treated?" 


Yeah. | very much agree with you about how to approach bias and also that, | called 
them structural interventions, right? It's really hard to change how people feel 
because people how we feel, but sometimes we can change what people do and 
that will change the impact. 


Well, | couldn't agree with you more. Atul Gawande wrote a book called, "The 
Checklist Manifesto." Fantastic book. His argument, he starts by talking ... It's about 
medicine mainly but he talks about how flying a plane used to be an art, a craft, 
right? You had these heroic pilots with their bomber jacket, and their scarf tossed 
back behind their shoulder. 100 years ago roughly, when they started putting some 
... [guess less than 100 years ago, when they started putting some structure in this, 
the pilots were really resistant. They were like, "You can't give me rules. I'm an 
artist. I'm an aviator. I'm brave. I'm a hero. You can't tell me how to fly my plane and 
it will take all the magic and splendor, and a good people won't want to do it." | 
said, "well, you know what? Just do the following. Before you land, make sure the 
landing gear is down and run through this list." As a result, planes don't crash. This 
kind of checklist can massively change human performance with it and behavior 
without having to change the belief system. It's really powerful. | agree 100% with 
you. 


| also want to believe because we definitely share this philosophy that those kinds 
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of interventions and checklist so to speak can lead toward belief structure changes 
as well. 


| hope so. 

Yeah. Well, it's the goal. Different ... 

You can't tell people that's the goal. Well, he can't, "Sshh." 
No, that's not the goal. 

Yeah. 


Yeah. No, | know. | have a different question about diversity and inclusion. When 
you started your diversity and inclusion work at Google, did you need to create a 
business case for investing in it? What did you do around that to get that case? 


No. 
No? 


No. For two reasons, one was it was something the founders always cared about. 
Even pre-dating me in the very earliest days. Sergey for example, always cared 
about for example, making sure there were women in the company than you would 
normally get. The first interim class Google ever had in '99 or 2000 was four people 
and Sergey, I'm told insisted that there would be two women and two men. One of 
those women or one of those men, one is Jen Fitzpatrick who now reports to 
Sandara and runs all the geo stuff and the other is Ben Smith who also reports to 
Sandara and runs a bunch of identity stuff but it was always important. Then, we 
had an eng leader, Alan Eustace who cared deeply about LGBT issues. We were late 
to the party in a lot of other areas, issues around the black community, issues 
around the Hispanic community. | should say it, not we, | should say | because | 
shouldn't be speaking for them. 


When it was time to really build out that infrastructure, there were two great 
things. One was at the most senior levels, the company cared about it and the 
second was one of the funny things is when you're an executive or a manager and 
you have teams of people under you, you do whatever you want with the head 
count. | always run this approach of feast or famine where for some of my teams, 
basically teams would get head count every other year. Just like | funded out 
analytics by not investing in development, | funded diversity by having fewer HR 
people that following year and it allowed me to actually add a lot of bodies. In that 
way, there were also, of course, you will ask for resources and so on but I had the 
luxury of not having to fight in the way people sometimes do it in other 
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organizations. 


That's really compelling. | didn't know that about the first interim class engine. 
That's really cool. Another big topic that comes up in diversity and inclusion is of 
course, recruiting and hiring. Sometimes | feel like it's all we talk about although | 
personally believe that culture and retention are at least as important, but this is a 
hiring question. You see a lot of programs discussion around diversity and 
representation of specific groups within the recruiting process and the hiring 
process. What about inclusion and thinking about how we include more types of 
diversity mentions, points of view perspectives in hiring? How do you do that? 


It's a great question. | think there's a few things. One of the things I've seen in my 
last job was that for people from a lot of different backgrounds, the tech industry is 
not a real welcoming place and preaching to the choir, but it's not the place people 
think of where they want to be. If you're black, that's obviously the case. In many 
companies, it's also the case if you're politically conservative especially these days, 
right? On the spectrum of conservatism, there's a wide range but if you believe that 
what you really want is people to bring their wholesales to work, you want the best 
people to bring their values and beliefs and you want to hammer that out and get 
the truth, you actually need some descending perspective and opinion. 


On the recruiting side, | think there's a couple of things. One is it's important to try 
to have your interviewed panels look like, feel like the people you're trying to 
recruit. That places an un do tax and burden on certain segments. Women in 
engineering, if you're trying to build an organization with lots of women and you 
want to improve a representation, you actually have to have women show up, 
totally unfair to the women because that asked them to do extra stuff and they're 
not compensated for it, they don't get a performance bump typically, so they 
should get some kind of recognition or reward for that or it should be valued by the 
organization. 


The second thing is | think it does get to your point about culture. You got to mean 
this stuff, you’ve got to believe it. It can't just be tokenism. It can't just be, "Oh, we 
want more more of one or other kind of person." One of the things | learned over 
the last decade was that ... Again, I'm late to the party but the sum of the minority 
is the majority. We did this piece of work where we looked at ... We had its program 
called GDNA, Google DNA. It's a longitudinal study. It's four or five years in. My goal 
was to have it run for 100 years but every six ... Hopefully we'll get there. It's model 
on the framing of [hard 00:25:18] study. Every six months, it's this incredibly 
detailed survey of people who have opted in, who have been randomly selected 
and then, they opt in. Then, we make sure the samples still represented it. There's 
former Google employees in the state as said well. 


One of the things we found in one of the survey cycles was that everybody feels like 
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they're part ... Everybody at that company and the alumni that are in the 
population, feel both like they're part of a community and they're completely alone 
and isolated. Take your stereotypical MIT educated, upper middle class, Caucasian, 
male, straight, pick all your majority groups. That person too has the experience of 
feeling alone and on the outside. In that experience that we all share and if you 
think about like | got picked on in school. I've had a lot of good, great things in my 
life but | don't have a bunch of really bad things, and I'm a refugee and an 
immigrant and there's all this religious noise in my background and all these things. 
Everybody has that kind of touchstone in their lives. 


When you come back to inclusion, it's not just having people show up in the 
interview process but when they interact with your company, they need to see that 
you actually seek out what is it that makes everyone feel a little bit like they're 
unique and special and alone, and that's the basis for actually finding commonality 
across every background, | think. 


Yeah. That's awesome. I'm actually flipping around because you just said that 
sparked a different question that | have in the different place. 


Of course. 


We are now, as of the last couple of years, enthusiast of surveying engagement and 
employee's sentiment. Within the diversity and inclusion program, we also last 
year, did a lot of research which generated a lot of ... It has quantitative data, but to 
date, that was a representation data. Then, we have all this qualitative data about 
what | would call experiences of inclusive or non-inclusive culture which we would 
like to quantify. We are going to do a survey experiments to do that, and | 
wondered what your experience is with quantifying qualitative human experience? 


Yeah, it can be done. You should do it. Yes. I'm in favor of it. 
Thank you. If we're going to. 


| mean, you're measuring perception but perception matters, right? The things to 
correlate it with are things like performance, progression. In your career, retention. 
| imagine you'll see differences most companies do between your different groups 
in the company. It's important then, | think to be open and honest about those 
differences. Again, that places people who are less happy in a difficult situation 
because you don't want every sort of meeting to be like I'm the one female 
engineer or l'm the one Hispanic engineer in a group of white men. Everyone is 
looking at me and now they know on average what ... You want to be thoughtful 
about if and how you have those conversations but my belief is if you have the kind 
of culture, | think you do based on the work product you have and the reason many 
of you seemed to be here, | think it's fertile ground to have these conversations. 
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It's important to track it overtime and the numbers are going to suck. They're going 
to be bad, and improvement is going to be slow and take a while but it's important 
and it's going to get better, then it's going to get worse. One of the things that 
happens ... Sorry to ramble. 


No, please. 


One of the things that happens is when you start talking about these issues, you 
see a spike in people saying, "Actually, it is bad." You see more complaints. As 
companies have talked more about women's experience in the workforce, you see 
women raising the issue of harassment a lot more. Some of that is just more 
awareness, some of it is more safety. It does not necessarily mean all of a sudden 
harassment in spike but when you look at it or when executives look at it, they go 
like, "Oh, this is really bad." 


This is part of the challenge universities have. Why universities are so careful and 
delicate in how they talk about rape on campus and harassment on campus and 
things like that. The greatest thing is to look at the data, share it but let people 
know there's all these dynamics without making excuses for it, right? Because when 
you're getting reported, it's real and people's sensitivity to things change too. They 
report something that they may not have reported. Overtime, it evens out in terms 
of people getting calibrated about what's a real issue, what's intended, what's not 
but it's all worthy of debate and discussion in transparency because overtime, you 
get to a place where it's an institution. You're able to actually talk about all these 
issues and work through them and have people to actually be treated better. 


Thank you. That's very much how | see it and we're hoping to use this kind of 
information to better target the programs that we're rolling out in experiments 
basically. 


Well actually on that note, it's great that you're going to have programs, | was going 
to say with them too because we did this work on... This is public as well, on team 
effectiveness and one of the five things that's the underpinning things that makes 
teams effective is psychological safety. Then, what we immediately did was we wind 
up and sort up all teams, we did this diagnostic, your team is not safe. The teams 
like had total knock downs. | was like, "Well, we knew it but now ..." This is really 
awkward. | don't know what to do and the manager's like, "You need to feel safe." 


When you're dealing with these kinds of issues, as you're doing and as you plan to 
do it, it's really important to have an intervention and support structure setup to 
deal with this but just know it's going to be hard. Last thing, I'll stop talking but for 
those who are not in the minority, however you slice it, in my view, it's your 
obligation to step up and moderate your behavior and lead and call people on 
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these kinds of issues. There's a lot that's been written about pick whatever your 
group is but here's how to succeed in business and society and what have you in my 
very, very strong belief. What I've seen is it's the people like me or who's 
superficially like me who actually have the responsibility for taking action and care 
and making place as safe. 


Trivial example, but | know this happens everywhere. Maybe not trivial. It's not 
unheard of. | am not saying it's common, but it's not unheard of if you're a black 
woman working in a corporate American environment to have a white person touch 
your hair at some point in your professional trajectory. | don't know if it's ever 
happened to you, I don't know but I've had lots of stories where someone would 
just reach out and touch the braids, and you don't actually touch other human 
beings like that. | wouldn't reach out and touch your hair but people do that, and 
my view is when that kind of thing happens, it's obviously like an awkward super 
crazy dynamic between these people and you walk away from it. I'm told, the 
women I've talked to have shared this with me and said they walk away or they're 
like, "What should | have done?" Or, "What's the right response?" 


The correct response is for the person who looks like me observing this happening 
going like, "Oh, what are you doing? That's not cool. Knock it off. That's weird." 
There's lots of ways to engage, but taking the onus off the people in that 
interaction and using the privilege that you have and the status that you have and 
the fact that you're not the person in the middle of it can defuse it and turn it into 
healthy conversation. 


| did not pay him to say that, but thank you so much because that is so right on. 
Okay. Last DNA specific question and about a different dimension than the others 
that we've talked about. We are a highly international company, organization 
project, we have staff in 17 or maybe more than 17 countries if you count contract 
stuff and then, community and many, many more. | know Google has also people in 
many, many countries. We're doing various things to bring ourselves together 
across the geographies and I'm interested in what Google did during your time 
there in that regard and what you think works or doesn't work for that kind of 
diversity? 


In terms of community building or in terms of diversity, or ...? 


In terms of how you build community in an organization that's across so many 
geographies? 


Yeah. There's two big levers to pull. One is in my experience, and so it's just one 
data set but one is when you open up new locations or when you have small 
existing locations, send someone from headquarters who's been in the 
headquarters for a really long time, who's known, who's trusted, who's a culture 
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carrier but also has network back in the headquarter because building intimacy 
across different regions is predicated on trust and quality of the network among the 
different people, the information flows. You want to see those situations with 
somebody who is known, trusted, revered, even if that's not ideal for that person to 
do it as an assignment. It's the best thing for the organization and then you grow to 
local folks. The second and this is expensive is travel. Bring people together, 
whether it's the whole company comes together and that works until you've got a 
couple of thousand or more people and then it gets a little tough in terms of 
intricacy. 


We're good right now. 
You're good? 
We're into that one. 


Also, get people from headquarters out to all the other offices and have the people 
in the different offices connecting. In people operations, one of the things | did was 
... [believe in it so strongly that we had an awards program. Lots of teams that 
were going to ... Right. Every year we pick our 10 people of best exemplified the 
values we cared about in people operations and the prize was you get to visit any 
two offices in the world, and you got to fly business class which was big deal like 
really, really big deal. Of course people would pick like, "Well, I'm going to go to 
Singapore and then I'm going to go to South Africa." 


Because they want to get the mileage and see the difference. The only requirement 
is in each of your offices, you do talk like this and meet people and have lunches 
with people and share what you're learning. When you come back, you document it 
and share back what you learned. It's really, really important because companies 
live and die by these human connections. The only way to have that is you got to go 
spend time with people. 


Yeah. | totally agree with that. Okay. | have a few more questions. Then, | want to 
open it up and we have some questions in the room and from the moderator. Are 
we good, Diane? Did you have a few more questions and then, switch? 


[inaudible 00:36:17]. 


Okay. | want to ask you one here, but I think that you addressed it in part earlier. 
We're so much smaller than Google like what? 170th approximately. I'm curious 
about what ideas you have about how we could run experiments and ways that 
take our size and agility into account? At the same time, we can struggle with that, 
an example would be we have a wonderful relatively new hire program that Diane 
spearheaded. When we started, we wanted to run an experiment but we didn't 
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really feel comfortable holding some of our staff out of that awesome experience 
especially being small. We didn't really give all the new hires the awesome 
experience, right? New hires who got the awesome experience. Is there any here? | 
don't know. | guess I'm wondering how you would recommend running 
experiments that don't adversely impact in a relatively small employee-based? 


Well number one, tell your engineers to cut you some slack because the standards 
of social science and the engineering are different in terms of statistical validity and 
things like that. The R squared is totally different. As you know, given your 
background doing all of these. 


That's true. 


We had a big back and forth where we're trying to figure out how to improve hiring 
and the R squared on what predicts who we should hire were like .3, .4. In people 
operations, we were like, "That's awesome," because you never get that in social 
sciences and the engineers were like, "If it's below .99 R squared, it's really 
non-predictive and we're not that impressed." Number one, cut yourself some 
slack. Number two, my opinion would be most of what we're doing is not life and 
death. Hold the population out and really see if there's a difference because 
otherwise, you don't really know. Now in a way, you have a controlled group which 
was everybody who went through it before, the new latest hires went through it so 
you can compare those two populations, there's a time lag effect. 


Yeah. 


You could do it that way, but don't worry about getting huge samples. Don't worry 
about perfect homogeneity between the different populations, but do absolutely 
try it in a spot first or for a finite duration and see how that works and then, 
compare it and don't feel guilty about the people who aren't getting the goodness 
because they also won't get the bad experiments that make them miserable. 


Cool. All right. | have one last follow on to that, and so think about your questions 
everybody and everybody out there in Air Mo land. My last one is when you do run 
a people experiment, how do you ensure that the groups are representative or 
representative of where you want to go? I'm not sure which. 


Well, the first thing is you have to ... First of all, it's ever perfectly represented, 
right? Because even if demographically it is, cycle graphically, it won't be. You could 
inadvertently have a whole bunch of introverts in your sample. You do the best you 
can, but what's helpful is before we published the diversity data that Google put 
out several years ago, we actually surveyed the whole company to gather that data 
and ask them to voluntarily share it. We made sure we actually have robust data at 
least on that dimension which then feeds in the experiments on an anonymized 


Mozilla Speaker Series - Laszlo Bock, Former SVP Google People Page 14 of 24 


Larissa Shapiro: 
Laszlo Bock: 


Larissa Shapiro: 


Laszlo Bock: 


basis. 


The reason we have to do that is because the way the federal government ... If you 
do work with the federal government, you're required to submit a form each year 
that has your diversity data and the way the government requires you to get it is 
when people get hired by a company, there's the employment form. It's optional 
whether or not they indicate the diversity. There's a few finite buckets they can 
slide into. A lot of the people don't fill that out. The government though requires 
you report on everybody. The best practice on this is HR departments and let speak 
specifically about Google but in general, HR Department will first look at someone's 
name and go, "Oh, does that sound like that person is Hispanic? Does it sound like 
they're from the Middle East?" If that's not helpful enough, they will often look in 
the employee database and look at their face and go like, "Hmm. Are they Indian or 
are they Hispanic? I'm not sure I'm going to say Indian, so they're Asian." 


That's the quality of data most company start with and report from. It's a leap of 
faith to get your employees actually tell you who they actually are but it does give 
you more validity. You get into a small end problem were for a lot of the slices, so | 
know when we've share data in Google publicly, one of the criticisms, the valid 
ones has been, "How come we don't slice the data in more ways?" Internally, 
people say the same thing too. While there's enough, for example, black people at 
Google to be able to slice some of the data, once you start doing job level analysis, 
what you end up with is like, "Okay. Well in Mountain View at this job level, in 
software engineering, how many do we have? Oh, it's one." You can't do the 
analysis. 


Certainly, it's Elmer, George or Nancy? 
Yeah. 


Yeah. Not cool. Yeah. We are familiar with this problem. | think the other thing 
about identification is super important. We have approached [inaudible 00:41:24] 
by just strongly encouraging our staff to report, but | don't know if everybody does. 
You might hear from us again encouraging you to do it, but it's challenging and also 
we're a privacy-oriented companies. | think sometimes folks just don't want to 
disclose, and we try not to guess if we don't have to. 


Well, we always tried to tell people what the information is going to be used for, 
who has access to it and what the controls are as you do with your user data, right? 
It also takes time. The first time, we did an all employee survey. The participation 
was 50%. Now, it's in the 90's but it takes time and the biggest thing is don't betray 
the trust along the way which of course, you wouldn't. The second is share the 
results back so that people can see, "Okay. Something good is coming of this. It's 
not just my overlords monitoring me," but it takes time as you know. 
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Yeah, it does. We just shared back some results from our most recent one in our 
last company meeting and | hope that people would agree that that's a useful 
practice. If you don't agree, we want to know. All right. That's the end of my official 
questions. Thank you. 

You're welcome. 

Diane, do you want to handle the additional questions and from the room? 

Yeah. 

We have a few keyed up from the moderator too. 

Oh, okay. Great. 

We have them, or | have them. Whatever. 

Yes. We have a number of questions. We will get someone in the room real quick. | 
want to take one through the global channel first. Lots of interest in this topic. First 
two questions come from someone in our hiring, very interested in this subject. I'm 
going to come mind them. The first one is what are one or two things that you wish 
you had done differently regarding your diversity hiring? Then related to that is did 


you have a checklist for de-biasing hiring? 


Oh, great questions. Before | answer, this is recorded and streamed and all of this, 
right? Non-Mozilla people see this? 


Correct. 

Yeah. 

Okay. Can | see that microphone? | did a double take. For people not in the room, | 
don't know. This is their microphone. It's the weirdest ... | don't want to call it speed 
back, but this is like ... 

We can throw it around. 

We throw it. 


Do you really? 


We will do it in a minute. 
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Okay. 

Throw it back. Threw it. Throw it to Diane. 
That's awesome. Okay. 

| get [stereo pitched 00:43:41]. 


Okay. The questions were hiring, did we have a checklist and what was the first part 
of the question again? I'm sorry. 


One or two things that you would have done differently? 
I've done differently? Oh. 
With diversity. 


Yeah. A lot of things. | would have focused on it way, way, way sooner. | could argue 
or hide behind the fact that, "Well, we were growing like crazy. We just wanted to 
hire as many people as we could," but | would have focused way, way, way sooner 
on it. Also, | think had | been able to would have invested way more in education 
and community outreach? There was something that got suppressed a couple of 
months ago. A number of years ago, the diversity team opened an after school 
program in Oakland and they didn't say it was a Google thing. It wasn't announced 
at Google as a Google thing. It wasn't filled with Google products. They actually 
went to Oakland. They went to a bunch of schools and said, "Well, it would be 
helpful." They said, "Well in this neighborhood, kids just need a place after school. 
Maybe they can get some tutoring." They showed up and started doing that. 


Three years later, it somehow like leaked that, "Oh, it was actually Google behind 
them." That kind of thing where you show up, you ask the community what they 
need, you provide it and you know what? Three or five or 10 years later, it turned in 
to something. | wish we've done more of that sooner. In terms of the checklist for 
interviewing, the new hire orientation and new manager orientation and interview 
training all go through ... All have unconscious bias training embedded in it. I'm not 
sure actually if it's as formal as a checklist, but there's a lot of things in there about 
just being aware of these differences, making sure you're creating and 
accommodating environment. There's obviously lots of dumb things you should not 
ask. If you're interested, actually if the Mozilla folks are interested, | can connect 
you with the person who's still at Google who can share whatever's in there. 


Yes, please. 


Thank you. 
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You can connect with me, and | would pass them. 
Oh, great. 

Awesome. 

Yeah. 

| think we had one in the room. 

Go for it. 

I'm not really supportive. 


If I'm maybe selfish and ask two questions, first being the more appropriate 
question. Is there a program or maybe a perk program that you are interested in 
launching at Google, but just didn't make sense for an organization of your size and 
scale? 


Yeah. It's interesting. There's a lot of programs like that. | don't know if the size was 
a constraint or not. For example, there was an article, | think New York Times 
Magazine this time, that talked about a healthcare chain, hospital chain in Texas, 
and what they were experimenting with was they were just sending nurses out to 
check on people. If you're diabetic, nurse will come to your house two or three 
times a week just to check in on you and see how you're doing. Are you taking your 
medication? Are you getting up? Are you moving around? What they found was a 
massively reduced cost because people comply better with treatment regimens and 
so on. 


People also open up and share a lot more of what they do. At one point, | suggest, 
"Why don't we just hire 30 nurses or contract with a hospital and just have them 
start going to every employee's house all the time?" That didn't get off the ground, 
but | think it would have been really powerful. We briefly had a sabbatical program 
which I think lasted two years and then that was cancelled. There's a whole bunch 
of things but there's not like one big shiny ... The beautiful thing was we were able 
to do a whole lot of crazy stuff and I'm sure they will continue to do so. Not a lot of 
regrets on that side. 


One more question. Early on, you mentioned being a fan of human nature and 
science. Are there particular books that focused on human behavior that you just 


gravitate towards or maybe are favorites of yours? 


Yeah. There's a couple that are very specific things and that | really like. There is a 
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book written in the 70s called Impro, I-M-P-R-O, about improvisa ... Well, okay. See, 
this is a special place. For those on video like, I've mentioned that book before and 
most of you are like, "What is that?" Awesome. There's this fantastic ... It's about 
how to improvise and there's this awesome, awesome chapter about masks and it 
was a real epiphany for me because the author who's that instructor writes about 
how he's trying to teach people how to act and how to improvise and how to 
inhabit different characters. It's really hard for people. 


Then, the exercise is he gives them masks and all this weird grotesque oversized 
masks and what have you. All of a sudden, they completely transformed and now 
just how they talk but their movement. They become hunched over. They can move 
faster or slower. They become these personas. The insight for me, there's lots of 
cool stuff in the book but the insight was we're massively constrained by how we 
think of ourselves and how we identify our identity when in reality, any of you can 
be anything, right? Anything at all, any kind of person at all. In a way, it's a choice. 
The mask because it mediates between my actual face and the world gives people 
this freedom. It sparked this exploration of what else can you do in institutions to 
free people from that kind of thing? 


| know Richard Dawkins, there's some controversy associated with him but his 
book, "The Selfish Gene," is also really fascinating because again, it's biology. It's 
not people stuff per se, but the dynamic around self-preservation and how 
organisms and components organisms perpetuate themselves actually has a lot of 
metaphor in how companies behave. Then, the last one is if you don't have a 
technical background, there's a book called, "Full House," that was written by 
Steven Jay Gould who | was in Evolutionary Biology at Harvard and that was my first 
exposure to statistics because he basically argues ... He's incredibly rationale and 
scientific. He opposes creationism sources that his whole book is constructed 
against the argument that there's infinite complexity surely that was designed by a 
master clockmaker and he says, "No," because if you think about statistics, there's a 
minimum level of complexity which is a single cell or a component of a cell. 


There's infinite complexity above that, so given long enough time, you're going to 
have lots and lots of things happen. Then, he goes through the statistics about why 
all that happens. Inevitably, you're going to have courage and evolution and you're 
going to have speciation and all this complexity. Those are the three books. Sorry 
that none of them are Good to Great or whatever. | didn't read that. I'm sure it's 
terrific. 


Those are great books. 
Okay. We have some more. 


Yeah. 
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All right. This is an interesting question. How do you maintain inclusion or inclusive 
environment when you have religious or political views that appeared to be in 
conflict with each other? 


It's super hard. It gets to the difference between behaviors and how they manifest 
in what people actually believe. It also gets to the issue of civility. The journey | 
experienced at Google was first, we started focusing on diversity. That led very 
naturally to this notion of inclusion and unconscious bias in particular as a way to 
unlock that because the problem with diversity per se is most people, as | said, 
don't identify as racist. There's a couple of people who were like, "Yep. | totally get 
it." There's a couple of people who are actually terrible people who are racist and 
filled with hate and will never get it. Then, everybody else is like, "Oh god, 
corporate training. Really, | have to ..." Unconscious bias is a way to get around that. 


Then what we saw and | saw was that that raised this kind of hard issues about you 
get more reporting, different segments of the population are less happy than 
others, how do you deal with that? The key there was we went through this 
multi-journey talking about civility and how do you have conversations about these 
issues? It's actually particularly an engineering organizations. It's a non-trivial thing 
to tackle because if you look at the academic research, there's five or six archetypes 
of bad behavior. One of which that you'd see a lot in this kind of settings is people 
think, "Well, you're talking about inclusion. I'm just being me. I'm bringing my 
whole self to work. Isn't that what you wanted?" The problem is inclusion isn't like 
a binary logical set of laws, it's a value system. 


There are certain views that every view, every debate is worth having if the other 
person is truly interested in the pursuit of knowledge of growing and getting better 
and if the terms are civil but it's really, really hard to get there particularly in 
engineering cultures, which price, finding the bugs and what's wrong with your 
argument, and I'm going to have glory if | find this little tiny thing. It's difficult. | had 
one conversation with somebody who was a fundamentalist Christian who said 
they were really uncomfortable with the company's focus on LGBTQ issues on the 
coverage of benefits for people who are domestic partners, and they thought it was 
just wrong and asin. 


We had a very nice 90-minute discussion about it and | argued, | ended up saying, 
"Number one, | think you're actually on the wrong side of history." Number two, "In 
any case, this is the company's perspective. This is what we're going to do. We're 
going to keep doing it. I'd much rather have you be part of the conversation here." 
The person was really thoughtful and lovely and open, but determined and 
resolute. If it causes you so much anxiety and pain and tension that you can't be a 
part of this that all the good that you do agree we're doing is all set by the fact that 
we have gender neutral bathrooms, then this may not be the right place for you 
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and that's okay. I'd much rather have that voice where we can have a civil 
conversation but the ability to have that conversation is everything. That 
pre-supposes that both side is actually want to explore or come to understanding 
or come to some norm of behavior. Unfortunately, at the extremes, people often 
don't. 


In those cases, someone is not willing to have a conversation. Then you fall back to, 
"Well, here are the norms." One of the cool things about the unconscious bias for 
me was it definitely change the norms in the company. That's not acceptable. 
There's lots of places you can be that aren't this one. 


Wow. We have a couple of more [crosstalk 00:55:19]. 
Thank you whoever ask that. 
Yes. Joe Fish. Yeah. I'm not playing football. I'm too nervous to play football. 


Hi. | have a specific question. l'm going to go and try to not make it too based on my 
own experience. | joined Mozilla a couple of months ago, three months ago now. At 
the same time, | was talking to Google and | had an offer from Google, and | had an 
offer from Mozilla. One of the things | noticed about the process of Google and | 
have talked to people there before and have gone to various stages of interview, 
and it seemed to be optimized for software engineers and I'm more of a computer 
interaction person. 


A lot of the processes ended up being aimed at this one thing. | think there's a 
larger example there which is that how do you avoid emphasizing the center of the 
bell curve, and not getting the people or if you want to figure it the other way, how 
do you make sure that either people who are down the end of your long tail, 
however you define that long tail end up having a good experience as well? 
Because that seems like a fundamental problem, right? You can get the majority of 
your people, but what do you do about those exceptions? What do you do with 
people, whether they are in a different domain, whether it's different skill level, 
whatever those extremes are. How do you deal with that problem? 


That's a great question. It's so funny like now, not being there because [inaudible 
00:56:49]. I'd apologize, I'd say like, "We'll work on it. We're good," but it's not my 
place to, so | was sort of like living this real time. | guess on the meat of your 
question, you work as hard as you can to make the process as good and 
accommodating and warm as you can. You may have gotten the survey afterwards 
that said like, "What do you think of the process?" That should be standard and 
everybody should be getting them. Nothing is 100% compliant or functioning 
perfectly. There's been a lot of work that Google did when I was there to focus on 
the candidate experience and make it better. There are some norms which are still 
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just really off footing like people taking notes on their laptops during interviews and 
not making eye contact and things like that. 


One is you ask people and then you take the feedback and try to improve the 
process. The data on that is that you'll never get to 100% of people saying that was 
awesome because at the end of the day, rejection is never awesome but the 
favorability ratings on, "Was | treated with respect. Did the process seem like it 
assessed me in a fair way?" All that's trended out dramatically from my own 
interview experience. | think the very point is in a large institution or probably at a 
small one, one of the challenges | always felt was any given year, you have five or 
10,000 people joining. Ideally, you want everyone to do everything perfectly, right? 
You don't want them to focus on pedigree, academic pedigree. You don't want 
them to focus on test scores. You don't want them to treat UX person or human 
interaction person like a software engineer. 


You've got some large number of people who before they interview should get 
trained, often they do, sometimes they don't. Sometimes the training sticks, 
sometimes it doesn't. They have a thousands of new people coming into the 
system, not only saying, "I don't know how to do this yet," but saying, "I do know 
how to do this because here's the way I've done it in every other company," and try 
to retrain them. There's a treadmill and you are on your way we're constantly trying 
to improve the quality of it. There's a couple of areas where and even today, we 
proved years and years and years ago that your academic performance is not 
predictive of your professional performance once you're more than two years out 
of school and yet, there's parts of the company where they still care a lot about it. 


| guess the best answer I can give is it's a work in process. | think the experience, | 
can say with confidence, I'm sure was better than it would have been five years 
ago. There's a lot of efforts made to try to make it more accommodating. 


My experience was better than when | talked to people five years ago. | just wanted 
to come to Mozilla instead of Google. When | came down and head to two office 
and look to them, Mozilla was better but ... 


Yeah. Now, it's like anything. When you're dealing with people, you can't engineer it 
perfectly because there's variation in the individual's human behavior. The big 
lesson is you can never stop working at the stuff. They can never stop in your 
priority because to your point, most of the people who have a human experience 
with Google have it through the recruiting side. Advertisers never talk to a human 
being rarely. Partners might need a few people but three million people apply to 
the company every year. You want everyone of those people to go like ... Most 
won't get hired. You have those people to say, "You know what? It was still a good 
experience." 
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Yeah. Yeah. All right. 
We're good. 
Are we good? 


Yeah. 


Thank you and thank you to everybody for coming and for your questions. Thank 


you so much, Laszlo for coming and sharing your insights with us. 


Thank you. Thank you again for all you do as Mozilla for the world. It's really, really 


important work. | appreciate it. 
Thank you. 

Thanks for having me. 

Yeah. 


Thank you. 


Yeah. Hang on. Diane, you have ... We have a gift to you. 


You can always remember us. 


Awesome. | love it. | love it. Oh, a teddy. Thank you. 


There's stuff in there, but this is a good bag. 
I can tell. Thank you, Larissa. 

Oh, thank you. 

Thank you very much. 

Thank you. 

Thank you. 

Awesome. 


Thanks everyone. 
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